109 research outputs found
Adversarial Unsupervised Representation Learning for Activity Time-Series
Sufficient physical activity and restful sleep play a major role in the
prevention and cure of many chronic conditions. Being able to proactively
screen and monitor such chronic conditions would be a big step forward for
overall health. The rapid increase in the popularity of wearable devices
provides a significant new source, making it possible to track the user's
lifestyle real-time. In this paper, we propose a novel unsupervised
representation learning technique called activity2vec that learns and
"summarizes" the discrete-valued activity time-series. It learns the
representations with three components: (i) the co-occurrence and magnitude of
the activity levels in a time-segment, (ii) neighboring context of the
time-segment, and (iii) promoting subject-invariance with adversarial training.
We evaluate our method on four disorder prediction tasks using linear
classifiers. Empirical evaluation demonstrates that our proposed method scales
and performs better than many strong baselines. The adversarial regime helps
improve the generalizability of our representations by promoting subject
invariant features. We also show that using the representations at the level of
a day works the best since human activity is structured in terms of daily
routinesComment: Accepted at AAAI'19. arXiv admin note: text overlap with
arXiv:1712.0952
Embarrassingly Simple MixUp for Time-series
Labeling time series data is an expensive task because of domain expertise
and dynamic nature of the data. Hence, we often have to deal with limited
labeled data settings. Data augmentation techniques have been successfully
deployed in domains like computer vision to exploit the use of existing labeled
data. We adapt one of the most commonly used technique called MixUp, in the
time series domain. Our proposed, MixUp++ and LatentMixUp++, use simple
modifications to perform interpolation in raw time series and classification
model's latent space, respectively. We also extend these methods with
semi-supervised learning to exploit unlabeled data. We observe significant
improvements of 1\% - 15\% on time series classification on two public
datasets, for both low labeled data as well as high labeled data regimes, with
LatentMixUp++
Filling out the missing gaps: Time Series Imputation with Semi-Supervised Learning
Missing data in time series is a challenging issue affecting time series
analysis. Missing data occurs due to problems like data drops or sensor
malfunctioning. Imputation methods are used to fill in these values, with
quality of imputation having a significant impact on downstream tasks like
classification. In this work, we propose a semi-supervised imputation method,
ST-Impute, that uses both unlabeled data along with downstream task's labeled
data. ST-Impute is based on sparse self-attention and trains on tasks that
mimic the imputation process. Our results indicate that the proposed method
outperforms the existing supervised and unsupervised time series imputation
methods measured on the imputation quality as well as on the downstream tasks
ingesting imputed time series
Efficient Continual Pre-training for Building Domain Specific Large Language Models
Large language models (LLMs) have demonstrated remarkable open-domain
capabilities. Traditionally, LLMs tailored for a domain are trained from
scratch to excel at handling domain-specific tasks. In this work, we explore an
alternative strategy of continual pre-training as a means to develop
domain-specific LLMs. We introduce FinPythia-6.9B, developed through
domain-adaptive continual pre-training on the financial domain. Continual
pre-trained FinPythia showcases consistent improvements on financial tasks over
the original foundational model. We further explore simple but effective data
selection strategies for continual pre-training. Our data selection strategies
outperforms vanilla continual pre-training's performance with just 10% of
corpus size and cost, without any degradation on open-domain standard tasks.
Our work proposes an alternative solution to building domain-specific LLMs from
scratch in a cost-effective manner
Using Clinical Notes with Time Series Data for ICU Management
Monitoring patients in ICU is a challenging and high-cost task. Hence,
predicting the condition of patients during their ICU stay can help provide
better acute care and plan the hospital's resources. There has been continuous
progress in machine learning research for ICU management, and most of this work
has focused on using time series signals recorded by ICU instruments. In our
work, we show that adding clinical notes as another modality improves the
performance of the model for three benchmark tasks: in-hospital mortality
prediction, modeling decompensation, and length of stay forecasting that play
an important role in ICU management. While the time-series data is measured at
regular intervals, doctor notes are charted at irregular times, making it
challenging to model them together. We propose a method to model them jointly,
achieving considerable improvement across benchmark tasks over baseline
time-series model. Our implementation can be found at
\url{https://github.com/kaggarwal/ClinicalNotesICU}.Comment: Accepted at EMNLP 201
- …